Stochastic Block BFGS: Squeezing More Curvature out of Data

is cheap (2), where Dt ∈ Rd×q and q min{d, n}. We employ three di erent sketching strategies: 1) gauss. Dt has standard Gaussian entries sampled i.i.d at each iteration. 2) prev. Let dt = −Htgt. Store search directions Dt = [dt+1−q , . . . , dt] and update Ht once every q iterations. 3) fact. Sample Ct ⊆ {1, . . . , d} uniformly at random and set Dt = Lt−1I:Ct ,where Lt−1L T t−1 = Ht−1 and I:Ct denotes the concatenation of the columns of the identity matrix indexed by a set Ct ⊂ {1, . . . , d}. 4. Block BFGS Update